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(57) Abstract: 

PURPOSE: To perform further accurate announcement of 
a surrounding state to a driver to effect proper 
decision during driving by a method wherein the image of 
an object, such as a preceding vehicle and an obstacle, 
having height is displayed without any distortion. 

CONSTITUTION: A photograph image at the periphery of 
a vehicle is inputted to an image input part 101. An 
input image is separated into a road region and a 
non-road region by a road detecting part 104. Only the 
road region is converted into a coordinate by a 
coordinate converting part 106 but the non-road area is 
not converted into a coordinate and parallel movement 
and enlargement/ contraction are effected by an image 
feed part 105. The two images are synthesized by a 
synthesizing part 107 and by displaying a result on a 
display part 103, an input image in the non-road region, 
such as an obstacle and a preceding vehicle, is 
recognized in a natural state. 
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(57)Abstract> 

PURPOSE: To perform further accurate 
announcement of a surrounding state to a driver 
to effect proper decision during driving by a 
method wherein the image of an object such as 
a preceding vehicle and an obstacle, having 
height is displayed without any distortion. 
CONSTITUTION: A photograph image at the 
periphery of a vehicle is inputted to an image 
input part 101. An input image is separated into 
a road region and a non-road region by a road 
detecting part 104. Only the road region is 
converted into a coordinate by a coordinate 
converting part 106 but the non-road area is not 
converted into a coordinate and parallel 
movement and enlargement/ contraction are effected by an image feed part 105. The 
two images are synthesized by a synthesizing part 107 and by displaying a result on a 
display part 103, an input image in the non-road region, such as an obstacle and a 
preceding vehicle, is recognized in a natural state. 
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CLAIMS 



[Claim(s)] 

[Claim l] A picture input means to picturize the situation of the circumference of 
vehicles and to input, and a road surface field detection means to divide the input 
picture from the aforementioned picture input means into a road surface field and a 
non-road surface field, A coordinate transformation means to transform the input 
picture from the aforementioned picture input means, and the non-road surface field 
extraction means which starts the picture of the non-road surface field separated by the 
aforementioned road surface field detection means, Circumference situation display 
equipment for vehicles characterized by providing a picture composition means to 
compound the picture in which coordinate transformation was carried out by the 
aforementioned coordinate transformation means, and the picture started by the 
aforementioned non-road surface field extraction means, and an image display means to 
display the synthetic picture by the aforementioned picture composition means. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Industrial Application] A body picture with the height of precedence vehicles, an 
obstruction, etc. is also distorted, and this invention detects the circumference situation 
of vehicles exactly as that there is nothing a display being possible, and relates to the 
circumference situation display equipment for vehicles which displays this 
circumference situation on accuracy more to an operator. 
[0002] 

[Description of the Prior Art] As circumference situation display equipment for vehicles 
in the former, there is "a circumference situation monitor for vehicles" currently 
indicated by JP, 3-99952, A. This changes two or more camera picture installed in 
vehicles into the picture seen from right above by inverse morphism shadow conversion, 
it displays it, compounding two or more pictures, and enables it to recognize the 
physical relationship of self vehicles and circumference environment enough to an 
operator. 
[0003] 

[Problem(s) to be Solved by the Invention] However, if it is in "the circumference 
situation monitor for vehicles" in the former as shown above Since it is changed about 
the display of an arrow, a zebra zone, etc. which were drawn truly on the body on a 
passage side, for example, a white line and a road surface, so that a front distance may 
serve as alignment in order to assume that it is what has all the bodies in a picture on a 
passage side and to perform coordinate transformation, although it becomes easy about 
grasp of distance Since a picture was distorted greatly and displayed to a body with the 
height of precedence vehicles, an obstruction, etc., there was a trouble that it was not 
easy to make the body on a display image and an actual body correspond. 
[0004] This invention is made in view of the above, a body picture with the height of 
precedence vehicles, an obstruction, etc. is also distorted, and it is displayed that there 
is nothing, and aims at telling accuracy about a circumference situation that a 
judgment at the time of operation can be exactly made to an operator more. 
[0005] 

[Means for Solving the Problem] A picture input means to picturize the situation of the 
circumference of vehicles and to input in order that this invention may attain the 
above-mentioned purpose, A road surface field detection means to divide the input 



picture from the aforementioned picture input means into a road surface field and a 
non-road surface field, A coordinate transformation means to transform the input 
picture from the aforementioned picture input means, and the non-road surface field 
extraction means which starts the picture of the non-road surface field separated by the 
aforementioned road surface field detection means, The circumference situation display 
equipment for vehicles possessing a picture composition means to compound the picture 
in which coordinate transformation was carried out by the aforementioned coordinate 
transformation means, and the picture started by the aforementioned non-road surface 
field extraction means, and an image display means to display the synthetic picture by 
the aforementioned picture composition means is offered. 
[0006] 

[Function] The circumference situation display equipment for vehicles concerning this 
invention inputs the image pckup picture of the circumference of vehicles by the 
picture input means, this input picture by the road surface field detection means It 
separates into a road surface field and a non-road surface field, and only a road surface 
field is transformed, coordinate transformation is not performed to a non-road surface 
field, but a parallel displacement, and expansion/reduction are performed, these two 
processing pictures by the picture composition means The input picture of non-road 
surface fields, such as an obstruction and precedence vehicles, is made to recognize in 
the natural state by making it compound and displaying on an image display means. 
[0007] 

[Example] Hereafter, one example of the circumference situation display equipment for 
vehicles concerning this invention is explained based on an accompanying drawing. 
Drawing 1 is the block diagram showing the outline composition of the circumference 
situation display equipment for vehicles concerning this invention, and consists of 
greatly the picture input section 101 which inputs the picture which picturized the 
circumference of vehicles, the image-processing section 102 which performs a 
predetermined image processing (detection, field specification, coordinate 
transformation, composition) to this input picture, and a display 103 which displays the 
image information of this circumference situation by which the image processing was 
carried out. 

[0008] Specifically, a two-dimensional display like TV monitor installed near the 
driver's seat is used as a display 103 as the picture input section 101 using the camera 
installed ahead [ body ]. 

[0009] Moreover, the image- processing section 102 is constituted by the following 
functional block. Namely, the road surface detecting element 104 which divides into a 



road surface field and a non-road surface field the picture as which the 
image-processing section 102 was inputted through the picture input section 101, The 
picture logging section 105 which starts the input picture in the non-road surface field 
separated by this road surface detecting element 104, It consists of the synthetic 
sections 107 which compound the logging picture from the coordinate transformation 
section 106 which transforms an input picture, and the above-mentioned picture logging 
section 105, and the picture after the coordinate transformation processing by the 
above-mentioned coordinate transformation section 106. 

[0010] Next, operation is explained. First, the picture of the circumference of vehicles is 
picturized by the picture input section 101 (camera), and it is inputted into the 
image-processing section 102. It is divided into a road surface field and a non-road 
surface field by the road surface detecting element 104 after this image information 
input. Moreover, although inverse morphism shadow conversion of the input picture is 
carried out by the coordinate transformation section 106, about a non-road surface field, 
display processing is performed in the state as it is in this case. In addition, these 
processings are performed by the picture logging section 105 and the synthetic section 
107. 

[0011] Drawing 2 is explanatory drawing showing the example of the input picture by 
the circumference situation display equipment for vehicles shown in drawing 1 , and a 
display image, drawing 2 (a) shows the input picture and drawing 2 (b) shows the 
display image, respectively. In above-mentioned drawing 2 (b), the hatching portion of 
the right-and-left triangle of the picture lower part is a portion in which an input 
picture does not exist, and the up hatching portion of precedence vehicles shows the 
portion which cannot be displayed since the picture of the front is hidden with 
precedence vehicles. 

[0012] Drawing 3 is a flow chart which shows image processing operation by the 
image-processing section 102 shown in drawing 1 , and drawing 4 is the display screen 
corresponding to each processing of above-mentioned drawing 3 . In addition, although 
explained taking the case of the picture ahead of vehicles, the same is completely said of 
display processing, such as the side, back or a method of a posterior, and a method of an 
anterior, here. 

[0013] First, it explains that overall processing flows. If processing is started, as shown 
in drawing 4 (a), the input of a color picture is performed through the picture input 
section 101 (S301), coordinate transformation of this input image information will be 
carried out by the coordinate transformation section 106, and it will serve as a bird's-eye 
view picture (S302). Furthermore, a color is recognized in the screen soffit section 



(5303) , and as shown in drawing 4 (b), the same color is extracted on the whole screen 

(5304) . Then, as shown in drawing 4 (c), expansion and contraction processing 
(after-mentioned) are performed (S305), labeling processing is performed to a non-road 
surface field (S306), and a boundary is detected for every field (S307). 

[0014] Next as shown in drawing 4 (d), straight-line detection is performed for every 
field (S308), and as shown in drawing 4 (e), the obstruction field on a screen is 
eliminated (S309). Then, logging processing and expansion, or reduction processing of 
the above-mentioned input picture to a picture is performed (S310), and it compounds to 
a bird's eye view picture by the synthetic section 107 (S31l). Then, as shown in drawing 
4 (£), drawing processing of a proper distance-between-twocars marker is performed 
(S3 12), and the image information which passed through a series of above-mentioned 
processings is displayed in a display 103 (S313). 

[0015] Furthermore, the above-mentioned processing is explained in full detail. First, 
processing operation of the coordinate transformation section 106 is explained. An input 
picture is set to A (x y), and it explains as what obtains B (i, j) by coordinate 
transformation. In addition, in order to make an understanding easy, the camera 
(picture input section 101) shall be horizontally installed to a road surface. When the 
height of the camera from a road surface sets the focal distance of H and a lens to F, the 
point on the road surface (it is assumed that it is level) which exists in the direction X of 
Yokote of a camera ahead [ Z ] is x=F-X/Zy=F-H/Z on a camera. ... (l) 
It is picturized by carrying out. 

[0016] At this time, the coordinate Z of the direction of the front on a road surface, and 
the coordinate X of the direction of Yokote and the coordinates i and j of a display image 
are made to correspond, respectively, and it is i=L-Xj=M*Z. ... (2) 

It shall be displayed that it becomes. However, L and M are suitable proportionality 
constants. Moreover, it is desirable that it is the relation of L>M in the display of the 
front or back. This is for what is necessary being just to display on a longitudinal 
direction about dozens of m to displaying to 100m place about the front. 
[0017] By the above-mentioned (l) formula and (2) formulas, it is x=F-M-i/(L-j). 
y=F-H*M/j ... (3) 

Becoming conversion is performed. Namely, B(i, j) =A (F-M-i/(L*j), F-H>M/j) 
If all the bodies currently picturized by the input picture by performing becoming 
coordinate transformation are the points on a road surface, the picture B G, j) which was 
seen from right above can be acquired. 

[0018] Next, processing operation of the road surface detecting element 104 is explained. 
In addition, separation with the road surface field and non-road surface field in this 



example shall be performed by the color picture. Moreover, the method of the 
obstruction detection currently indicated by for example, the 
Japanese-Patent- Application-No. No. 90043 [ three to ] official report may be used for 
this portion. By using a color picture, it becomes detectable [ a more exact non-road 
surface field ]. 

[0019] First, a color is recognized in the soffit portion of a screen after the input of a 
color picture. Since the soffit portion of this screen corresponds just before vehicles, its 
probability which is a road surface is very high. Next, the portion of the same color as 
the screen soffit section is extracted on the whole screen. A portion with the color of this 
appearance of this is a road surface field, and other portions are recognized as a 
non-road surface field. 

[0020] Specifically, when the color picture shall be expressed with the three primary 
colors of R (red), G (green), and B (blue), it is V1=R/(R+G+B), for example. 
V2=G/(R+G+B) 
V3=B/(R+G+B) 

What is necessary is just to calculate the degree of similar (inner product) by 
considering that the above VI, V2, and V3 is vectors, after changing. 
[0021] Namely, VI, V2, and V3 which become criteria in a screen soffit portion are 
calculated. Next, VI, V2, and V3 are similarly calculated by each pixel of a screen, the 
inner product of VI, V2, and V3 used as criteria is calculated, and binaryized 
processing is performed by a certain threshold, and let the pixel more than this 
threshold be a road surface field. Moreover, in order to use the color information divided 
into the brightness, the hue, and saturation other than the above, a hue and saturation 
may be considered to be the elements of a vector and the same processing as the above 
may be performed. 

[0022] Moreover, although you may carry out to an input picture, you may perform 
detection of a road surface field to the above-mentioned picture (B (i, j)). Since it is more 
effective to carry out to Picture B (i, j), it explains as what performs detection of a road 
surface field to Picture B (i, j) below. 

[0023] In the above-mentioned processing, since things other than the road surface color 
drawn on road surfaces, such as a white line, are also recognized as a non-road surface 
field, by carrying out expansion processing of the road surface field by the binary image 
processing, a white line portion is once changed into a road surface field, and 
contraction processing is performed further. Narrow lines and noise components, such 
as a white line, are absorbed to a road surface field by this expansion and two 
processings of contraction. Since obstructions, such as precedence vehicles, have a 



certain amount of size, the original size is held even if it performs expansion / 
contraction processing. 

[0024] In addition, since the distant obstruction is picturized small, although it may be 
changed into the road surface field by expansion / contraction processing by the input 
picture, since the distant place is also changed as a big body by using the picture after 
coordinate transformation, being absorbed by the road surface field is lost. 
[0025] Moreover, in the above-mentioned case, a distant body is reflected in the screen 
upper part, and conversion which is expanded, so that it becomes the screen upper part 
by coordinate transformation is performed, therefore, a thing with small one small since 
the body small reflected in the input picture since the field of the screen upper part was 
greatly judged to be nature is located far away and body itself - that judgment becomes 
unnecessary especially 

[0026] Furthermore, the above-mentioned expansion / contraction processing is 
explained in detail. Drawing 5 is explanatory drawing showing this expansion / 
contraction processing. First, in a binary picture, as shown in drawing 5 (b), expansion 
processing will be processed as "1", if at least one of 8 pixels near a certain attention 
pixel is "1." That is, OR logic of ai in drawing 5 (a) is taken. In addition, less than 
3-pixel "0" fields disappear. 

[0027] Moreover, as contraction processing is shown in drawing 5 (c), all of an attention 
pixel and a total of 9 pixels of 8 pixels of near process as "1" only at the time of "1", and, 
in other cases, it processes as "0." If only the number of times of expansion is contracted, 
the field which did not disappear will become the original size mostly. 
[0028] In addition, about a road surface field, "1" and when processing a non-road 
surface field as "0", it becomes expansion and contraction, and when reverse, the same 
result can completely be obtained by performing processing in order of contraction > 
expansion. 

[0029] Moreover, there are many bodies which have a perpendicular and a level edge 
component in an obstruction like for example, precedence vehicles, and especially the 
boundary portion with a road surface field serves as a perpendicular and a level straight 
line in many cases. Therefore, labeling processing performs field separation for a 
non-road surface field first. Then, in each non-road field, a boundary portion with a road 
surface field is detected as an edge. Next, a straight line is fitted to this boundary 
portion, and an obstruction field is decided. Moreover, straight-line conformity in this 
case is performed by applying three straight lines to one field. 

[0030] The 1st straight-line conformity detects a straight line with a horizontal 
inclination. On the boundary of a non-road surface field and a road surface field, the 



soffit of obstructions, such as precedence vehicles, has a part for a level bay in many 
cases. Therefore, if the detected straight line is made into a straight line 1, j=b can 
express a straight line 1. 

[0031] Moreover, the 2nd and 3rd straight-line conformity is for recognizing the right 
end and left end of an obstruction. In an input picture (A (x y)), although both the right 
ends and left ends of an obstruction have a perpendicular straight- line component, they 
are changed into the straight line prolonged in a radial from a view (zero) by Picture B (i, 
j). That is, the number of the boundary point on i=aj (edge point) is counted. It is aL 
about the slope of a line by the side of the leftmost where a is changed to a certain 
within the limits, and the number of counts of an edge point exceeds a threshold. It 
carries out and is aR about the slope of a line of most right- hand side. It carries out and 
considers as both straight- line detection results. 

[0032] Thus, a straight line is detected using each non-road field. In addition, in the 
picture of the front or back, the soffit portions of precedence vehicles or consecutive ness 
vehicles have high possibility of being a level straight line on a screen, the display of the 
side -- setting - the installation angle of a camera - the side — since a vehicle is not 
necessarily picturized by the picture waterworks common, a straight line 1 should just 
detect not detection of a level straight line but a straight line with a certain defined 
angle 

[0033] Next, logging processing of a non-road surface field is explained. First the 
non-road surface field in Picture B (i, j) is eliminated before logging processing. Since 
the point on a distant road surface is hidden with precedence vehicles etc. from the body 
with which this has the height of precedence vehicles etc., it is because it cannot image. 
Therefore, in order to aim at improvement in conspicuousness, about the portion hidden 
the account of a top, it eliminates by the color of fixation (painting out). A legible picture 
can be acquired by smearing away using the road surface color detected in the screen 
soffit section. 

[0034] Moreover, elimination in the above performs about the interior of three straight 
lines detected in each field. That is, elimination performs painting-out processing for 
the range used as aL, j<i<aR, and j in j>b. 

[0035] Next, an input picture is started in a non-road surface field. It is the lowest edge 
of the picture A (x y) which yD =F*H-M/b starts from the above-mentioned (3) formula. 
Moreover, a left end is set to xL =F-M-aL/L and, on the other hand, a right end is set to 
xR =F*M-aR/L. 

[0036] In addition, a upper limit is taken as a suitable fixed value. For example, xL-xR 
It is good also as width of face (height) of the direction of y of the field which starts a 



number of times. Thus., a square field is started from Picture A (x y). 
[0037] Next, processing operation of the synthetic section 107 is explained in full detail. 
Enlarging or contracting is performed and transmitted to the field which performed 
painting out within Picture B (i, j) for the field of the square started as mentioned above. 
By Picture A (x y), this is because it becomes the same size by Picture B (i, j), although a 
distant obstruction is picturized small. Since iL =aL, biR =aR, and b are the soffits of 
the obstruction field in Picture B (i, j) when it considers as j=b from a straight-line 
conformity result, it is iRiL. xRxL What is necessary is just to expand or reduce so that 
it may become the same size. In this way, processing which inserts in Picture B (i, j) the 
picture started so that a point (xR and yD) and a point (iR, b) might be in agreement is 
performed. 

[0038] And a circumference situation is displayed by displaying Picture B G, j). 
Furthermore, it is also possible by adding one horizontal line to Picture B (i, j) to display 
the proper distance between two cars, j shaft of Picture B (i, j) corresponds to the 
distance of the direction of the vehicles front. What is necessary is to find the proper 
distance between two cars from the vehicle speed, and just to draw a horizontal line on j 
corresponding to the distance. It becomes possible from the position on the picture of the 
position of a horizontal line, and precedence vehicles to grasp the distance to precedence 
vehicles easily 

[0039] Thus, in this example, since a screen can be seen and an obstruction can 
recognize easily by enabling it to display the picture seen from the camera about the 
non-road surface field in the state as it is in case projective transformation of the input 
picture is carried out to the picture seen from right above, operation judgment of 
braking, operation, etc. can perform exactly, for example, smooth vehicle warehousing 
operation etc. is attained. 
[0040] 

[Effect of the Invention] As explained above, according to the circumference situation 
display equipment for vehicles concerning this invention By dividing an input picture 
into a road surface field and a non-road surface field, transforming only a road surface 
field, not performing coordinate transformation to a non-road surface field, but 
performing a parallel displacement, and expansion/reduction, and making these two 
processing pictures compound A body picture with the height of precedence vehicles, an 
obstruction, etc. is also distorted, and it writes that there is nothing and a display is 
possible, and a body picture with the height of precedence vehicles, an obstruction, etc. 
can also be distorted, it can display that there is nothing, and accuracy can be told more 
about a circumference situation that a judgment at the time of operation can be exactly 



made to an operator. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing l] It is the block diagram showing the outline composition of the 
circumference situation display equipment for vehicles concerning this invention. 
[Drawing 2] It is explanatory drawing showing the example of the input picture of the 
circumference situation display equipment for vehicles shown in drawing 1 , and a 
display image. 

iDrawing 3] It is the flow chart which shows image processing operation of the 
circumference situation display equipment for vehicles shown in dra wing 1 . 
[Drawing 4] It is explanatory drawing showing the display screen corresponding to each 
processing operation of the flow chart shown in drawing 3 . 

[Drawing 5] It is explanatory drawing showing expansion / contraction processing of the 
circumference situation display equipment for vehicles shown in drawing 1 . 
[Description of Notations] 

101 Picture Input Section 

102 Image -Processing Section 

103 Display 

104 Road Surface Detecting Element 

105 Picture Logging Section 

106 Coordinate Transformation Section 

107 Synthetic Section 
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